libdispatch - event handling 


• Grand Central Dispatch 

• Asynchronous & concurrent programming 
model 

• From apple# 

• http://libdispatch.macosforge.org/ 


chenj@lemote.com 


What is event handling 

Example: event handling in glib 

• Create a GMainContext 

• Attach one or more GSources 

- GSource: wrap event and callback 

• GSource:pollfd-» 1:n 

- Built-in GSource: 

• timeout source 

• child watch source 

• idle source 


• Create a GMainLoop associating with 
GMainContext, then "g_main_loop_run" 












Priority: High ^ Low 



source list 


GSource 


GSource 


GSource 


Prepared 


Travel through sourcejist: 

■ Skip blocked sources 

■ Run prepare methods of 'not ready' sources , 
mark sources as ready if prepare returns true 

■ If encounter a ready source, 
set GMainContext->timeout to 0 

■ Only visit sources with priority no lower than 
highest priority of ready sources 

■ Return highest priority ready sources 



If can't find any ready sources: 

■ Set GMainContext->timeout the min timeout value 
of all sources(returned by prepare method) 

preparer 


UlOpClLV^I 



Initial[n] 





























pollfd pollfd pollfd 


cached_polLarray —i 

int poll(struct pollfd *fds, nfdsj nfds, int timeout) 


struct pollfd { 
int fd; 
short events; 
short revents; 


/* file descriptor */ 

/* requested events */ 
/* returned events */ 



Initial[n] 




















oil 


source list 



Travel through sourcejist: 

■ Skip blocked sources 

■ Run check methods of 'not ready' sources, 
mark sources as ready if check returns true 

■ If encounter a ready source, 

add it to pending_dispatches array. 

■ Only visit sources with priority no lower than 
\ highest priority of ready sources 












pending_dispatches 


^ I 


GSource 


GSource 


Travel through pending_dispatches: 

For each source: 

1. Remove ready flag 

2. If isn't a can_recurse source, block it 

3. Mark as in_call 

4. Link to a list which lives in stack: 
GMainDispatch->dispatching_sources 

5. Run dispatch, destroy the source later if returns false 

6. Cleanup: 

1> Unlink from GMainDispatch->dispatching_sources 
2> Restore the in_call state 
3> Unblock if is blocked 

pi cjpcil 



Initial[n] 























Event handling in glib 


Ca 11 g_main_ contextjteration o r 
g_main_loop_run in callback s callback: 

• can_recurse GSource 

Running callback will slow down event 
handling - Long running callback leads to bad 
responsiveness 



Event handling in libdispatch 


dispatch_source_t: wrap event and callback 

• source:kevent n:1 

• Set a event callback and (optional) cancel callback 

• Set source's target queue - callback runs in target 
queue 

• Source's priority is determined by its target queue 
Monitor event and do early process in mgr queue 

• In a single thread of the highest priority thread pool 

• Based on kevent 

• No priority in monitoring and early processing stage 



Event handling: libdispatch vs glib 


libdispatch has better responsiveness 

• Running callback will not slow down event handling 

• kevent is more efficient than poll 

glib can run nnulti-GMainContexts in nnulti- 
threads 



Create/Setup source 


dispatch_source_create 

• dispatch_source_t inherits from dispatch_queue_t 

• Specify source's type - dispatch_source_type_data_add , 

DISPATCH SOURCE TYPE DATA OR , DISPATCH_SOURCE_TYPE_MACH_ 
RECV . DISPATCH_SOURCE_TYPE_MACH_SEND , DISPATCH_SOURCE_T 
YPE PROC , DISPATCH SOURCE TYPE READ , DISPATCH SOURCE TYP 
E_SIGNAL, DISPATCH_SOURCE_TYPE_TIMER. DISPATCH_SOURCE_TYP 
E VNODE . DISPATCH SOURCE TYPE WRITE 

• Created in SUSPEND state 

dispatch_source_set_event_handler_f 



Run source(Installation stagel) 


dispatch_resunne 

• _dispatch_wakeup 

- send source to its target queue 

Invoke source in target queue 
(_dispatch_queueJnvoke) 

• _dispatch_sourceJnvoke ^ redirect to mgr queue 

Wake up mgr queue {_6\spatch_\Nakeup) 



Wake up mgr queue 


dispatch_wakeup(&_dispatch_mgr_q) 


- dx_probe 



^ if (!_dispatch_trylock(dou._do)) 
return NULL; 





Wake up mgr queue 


.dispatch_wakeup(&_dispatch_mgr_q) 


dx_probe 


_dispatch_mgr_wakeup 


if (!_dispatch_trylock(dou._do)) 
return NULL; 


dispatch update kq(const struct' 



.filter = EVFILT_USER 
fflags = NOTE_TRIGGER 













Wake up mgr queue 


dispatch_wakeup(&_dispatch_mgr_q) 



kevent{kq, &kev 1, &kev, 1, NULL); 
















Wake up mgr queue 


.dispatch_wakeup(&_dispatch_mgr_q) 


dx_probe 


_dispatch_mgr_wakeup 


if (!_dispatch_trylock(dou._do)) 
return NULL; 


_dispatch_update_kq(const struct' 



.filter = EVFILT_USER 
fflags = NOTE_TRIGGER 


dispatch_get_kq 


once 



.flags I = EV_RECEIPT 
kevent(kq, &kev 1, &kev, 1, NULL); 


-►Create kqueue 

-►Set monitoring EVFILT_USER 

-►Send mgr queue to its target queue 



















Wake up mgr queue 


.dispatch_wakeup(&_dispatch_mgr_q) 


dx_probe 


_dispatch_mgr_wakeup 


if (!_dispatch_trylock(dou._do)) 
return NULL; 


_dispatch_update_kq(const struct' 


dispatch_get_kq 


once 


-.flags I = EV_RECEIPT 
kevent(kq, &kev 1, &kev, 1, NULL); 



.filter = EVFILT_USER 
fflags = NOTE_TRIGGER 



Create kqueue 

Set monitoring EVFILT_USER 




















Invoke mgr queue 



-►Create kqueue 

-►Set monitoring EVFILT_USER 

>Send mgr queue to its target queue 11 ^^^^^_dispatch_queueJnvoke 




Invoke mgr queue 


dispatch_queue_invoke 

-► _dispatch_queue_trylock 
-►dx invoke .► 



_dispatch_run_tinners 

I 

_dispatch_get_next_tinner_fire( Sctimeout) 
I 

/* select workaround stuff */ 

kevent(_dispatch_kq, NULL, 0, 
kev, 1, timeoutp); 
















Event loop - running timers 

Process timer sources 

Two types of timer: wall timer; abs 
timer 

Link timers of the same type 
together(by order of expiration time, 
early ^ late) 

For each timer list: 

• For each expired timer: 

- Early process 

- Update next expiration time 

- Adjust its position in list 

- Wake up it 









Event loop - 





_dispatch_run_tinners 

_i_ 


_dispatch_get_next_timer_fire ■ 


Calc timeout 


Final timeout is 0 if exist an 
expired timer. 

Or calc by travelling through 
two timer lists: 

1. Find the earliest expiration 
times of all valid timers(not 

SUSPEND and has set expiration time) 

for each timer list 

2. Calc timeout from expiration 
time 

3. Convert abs timeout to wall 
timeout, find an smaller one 








Event loop - kevent 



_dispatch_run_tinners 

I 

_dispatch_get_next_tinner_fire 

I 

/* select workaround stuff */ 

_ ± _ 

kevent(_dispatch_kq, NULL, 0, 
kev, 1, timeoutp)', 


A generic method of kernel event 
notification on BSD. 

Usage: 

• int kq = kqueueQ 

• int ret = kevent(int kq, 

struct kevent *changelist, 
int nchanges, 
struct kevent *eventlist, 
int nevents, 

const struct timespec *tinneout); 









struct kevent 


Struct kevent 


uintptrj 

ident; 

Value used to identify this 
event. The exact interpretation is 
determined by the attached filter, 
but often is a file descriptor. 

int16_t 

filter; 

Identifies the kernel filter used to 
process this event. 

uint16_t 

flags; 

Actions to perform on the event. 

uint32_t 

fflags; 

Filter-specific flags. 

intptrj 

data; 

Filter-specific data value. 

void 

*udata; 

Opaque user-defined value 
passed through the kernel 
unchanged. 


filters 


• EVFILT_READ Data available for read on 

monitored fd 

• EVFILT_WRITE Buffer available for write on 

monitored fd 

• EVFILT_VNODE Notify operations on inode 

of monitored fd 

• EVFILT_PROC 

• EVFILT_SIGNAL 

• EVFILT TIMER 



Event loop 


_dispatch_run_tinners 

I 

_dispatch_get_next_tinner_fire 

I 

/* select workaround stuff */ 

I 

kevent(_dispatch_kq, NULL, 0, 
kev, 1, timeoutp)', 


process event 



Yes 


























Event loop - 


_dispatch_run_tinners 

I 

_dispatch_get_next_tinner_fire 

I 

/* select workaround stuff */ 

I 

kevent(_dispatch_kq, NULL, 0, 
kev, 1, timeoutp)', 


process event 


kevent 






kevent kevent 




^.filter == EVFILT_USER 

• Self-triggered event 

• Notify to update 
event-monitoring 
configuration 




















Event loop - 


_dispatch_run_tinners 

I 

_dispatch_get_next_tinner_fire 

I 

/* select workaround stuff */ 

I 

kevent(_dispatch_kq, NULL, 0, 
kev, 1, timeoutp)', 


process event 


kevent 






kevent kevent 




^.filter == EVFILT_USER 

• Self-triggered event 

• Notify to update 
event-monitoring 
configuration: 


nstall source 




















Run source(Installation stage2) 





















Run source(Installation stage2) 





















Run source(Installation stage2) 

dispatch_queue_drain 


Run source(Installation stage2) 


dispatch_queue_drain 

dispatch_queueJnvoke((dispatch_queue_t) source) 





Run source(Installation stage2) 


dispatch_queue_drain 

dispatch_queueJnvoke((dispatch_queue_t) source) 


^ dx_invoke 



• ds->ds is installed = true 


• Try to merge monitored kevents (each 
kevent can be identified by filter and 
ident) 

• if needs update, call: 


r 







Run source(Installation stage2) 


dispatch_queue_drain 

dispatch_queueJnvoke((dispatch_queue_t) source) 


^ dx_invoke 



• ds->ds is installed = true 


• Try to merge monitored kevents (each 
kevent can be identified by filter and 
ident) 

• if needs update, call: 

i 


kevent(...) 








Event loop 


_dispatch_run_tinners 

I 

_dispatch_get_next_tinner_fire 

I 

/* select workaround stuff */ 

I 

kevent(_dispatch_kq, NULL, 0, 
kev, 1, timeoutp)', 


process event 


kevent 






kevent kevent 





.filter == EVFILT_USER 

7 



















Event loop 


_dispatch_run_tinners 

I 

_dispatch_get_next_tinner_fire 

I 

/* select workaround stuff */ 

I 

kevent(_dispatch_kq, NULL, 0, 
kev, 1, timeoutp)', 


process event 
























Event loop 


_dispatch_run_tinners 

I 

_dispatch_get_next_tinner_fire 

I 

/* select workaround stuff */ 

I 

kevent(_dispatch_kq, NULL, 0, 
kev, 1, timeoutp)', 


process event 

























Event loop 


_dispatch_run_tinners 

I 

_dispatch_get_next_tinner_fire 

I 

/* select workaround stuff */ 

I 

kevent(_dispatch_kq, NULL, 0, 
kev, 1, timeoutp)', 


process event 



• Early process 

• Wake up source 




























Process event(early) 


Classify events: 

• level: ds_pending_data = kev->data 

- DISPATCH_SOURCE_TYPE_READ 

- DISPATCH_SOURCE_TYPE_WRITE 

• adder: ds_pending_data += ke->data 

- DISPATCH_SOURCE_TYPE_SIGNAL 

- DISPATCH_SOURCE_TYPE_TIMER 

• or: ds_pending_data | = (kev->fflags & ds- 
>ds_pending_data_nnask) 

- DISPATCH SOURCE TYPE VNODE 



Process event 


Wake up source ^ send to its target queue 


dispatch_queue_drain 

dispatch_queueJnvoke((dispatch_queue_t) source) 


L- dxjnvoke 





d s_h a n d I e r_f u n c(d s_h a n d I e r_ctxt); 





END 


